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Abstract 

The  aim  of  the  work  is  to  examine  the  possibility  of  developing  a  tool  to  track,  monitor  and 
predict  large  complex  system  development  by  enhancing  the  clarity  of  how  decisions  at  lower 
levels  impact  on  the  goals  of  the  project.  The  approach  uses  Maritime  Operations  Division's 
(MOD)  established  ability  in  combat  system  performance  modelling  using  MBSE  and  attempts 
to  connect  that  level  to  Operational  Capabilities  and  hence  Strategy. 

The  paper  leverages  off  MBSE  tool  capabilities,  developments  such  as  the  Whole  of  System 
Architecture  Framework  (WSAF)  and  research  approaches  such  as  the  Aligned  Process  Model 
(APM).  The  large  complex  project  examined  in  this  experiment  is  the  Future  Submarine 
project  due  to  the  authors'  experience  with  the  project,  however  any  other  large  complex 
project  would  have  been  equally  viable  for  the  experiment. 
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The  Challenge 

To  support  the  Decision  Maker,  we  want  to  look 
at  the  possibility  of  developing  a  tool  to 
monitor  large  complex  system  development 
by  enhancing  the  clarity  of  how  decisions  at 
lower  levels  impact  on  the  goals  of  the 
project. 
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STANDARD  SYSTEM  ENGINEERING  V’  DIAGRAM 
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Whole-of-System  Analytical  Framework 


WSAF  Model,  K.  Robinson,  W.  Power,  D.  Tramoundanis,  et  al,  WSD,  David  Harvey,  Paul  Logan,  2012 
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Capability  Development  Life  Cycle  -  Responsibilities 
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Figure  1-2:  Capability  Systems  Life  Cycle  -  Responsibilities 

Defence  Capability  Development  Handbook.  Commonwealth  of  Australia.  201 1 


Risk 

Increasing 


The  Challenge 


Three  reasons  why  this  could  get  difficult 

1 .  Developing  a  tool 

2.  Applied  to  large  complex  system  development 

3.  Attempts  to  enhance  the  clarity  of  how  decisions  at 
lower  levels  impact  on  the  goals  of  the  project. 
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Rationale  for  the  Proposed  Tool 

Requirement:  Quantify  how  low  level  decisions  impact  on  the 
goals  of  the  Project. 

When:  During  acquisition  phase  of  Capability  Development 
Lifecycle. 

Why  Not  Done  Now:  complexity,  cost  and  delay. 

DSTO  advice  needs  to  be  timely,  accurate  and  independent 


Tool  Requirements 

1.  Fast  and  automated,  1  week  turnaround  for 
advice, 

2.  Run  with  a  minimum  of  manual  effort, 

3.  Works  across  the  entire  MBSE  Project  database 

4.  Deliver  results  in  formats  readily  understood  by 
decision  makers 

5.  Staffing  limited  for  tool  development  and 
application 
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Approach  to  Enhancing  the  Clarity  of  Low  Level 
Decisions  on  the  Goals  of  Large  Complex  Projects 

1 .  Project  Goals  measured  by  submarine’s  ability  to  meet 
Top  Level  Requirements. 

2.  Achievement  of  Top  Level  Requirements  tested  by 
submarine  behaviour  within  agreed  defined  scenarios  and 
vignettes. 

3.  Submarine  behaviour  captured  by  executable  functional 
chains  containing  probability  distributions  and  analytical 
expressions. 

4.  Therefore  measuring  whether  Project  Goals  are  being 
met  can  be  tested  by  executing  submarine  functional 
chains  within  scenarios  and  vignettes  defined  by  the  Top 
Level  Requirements. 


Approach  Informed  by  Work  in  Other  Types  of 

Warfare 

1 .  Mine  Warfare  Command  Tactical  Decision  Aides 
Calculated  effect  of  low  level  changes  on  MCM  Task 
Group  Operations.  Used  Monte  Carlo  simulations, 
analytical  expressions,  and  probability  theory. 

Must  be  calculated  every  task  cycle. 

2.  Maritime  Air  Defence  Combat  System  Performance 
Prediction  using  MBSE. 

Calculation  time  twelve  hours  once  models  built. 
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White  Paper  Strategic  Roles  of  FSM 


DEFENCE  WHITE  PAPER  2009: 

Chapter  9  p70 

The  Future  Submarine  will  be  capable  of  a  range  of  tasks 
such  as; 

1 .  Anti-ship  warfare; 

2.  Anti-submarine  warfare; 

3.  Strategic  strike; 

4.  Mine  detection  and  mine-laying  operations; 

5.  Intelligence  collection; 

6.  Supporting  special  forces  (including  infiltration  and 

exfiltration  missions); 

7.  Getting  battlespace  data  in  support  of  operations. 


Impact  of  High  Level  Function  Failure  on  Project  Goals 
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Tool  Implementation  -  a  possible  approach 

1.  One  complete  high  level  function  failure  is  not  likely 

2.  Reality  is  marginal  performance  changes  in  some 
functions 

3.  Approach  for  tool  construction:  functional  chains 
executing  scenarios  and  vignettes  with  MBSE 

4.  Functions  incorporate  external  information:  analytic 
expressions,  tables,  graphs,  probability  distributions 
etc. 

5.  MBSE  Model  execution  tightly  connected  to 
Operational  Requirements,  Architecture  and  System 
Engineering  database. 

•  Removes  translation  errors  between  models 

•  Enables  cross  referencing  within  MBSE  database 


Illustration  of  Designed,  Marginal  and 
Failed  Functional  Behaviour 


FI  F2  F3  F4  F5  FI  F2  F3  F4  F5  FI  F2  F3  F4  F5 


Designed  functional 
performance 

I  Required  parameter  values 


Marginal  functional 
performance 


Failed  functional 
performance 
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Mine  Warfare  Modelling  -  Levels  of  Abstraction 


Level 

1  Type  of  model  | 

1  Characteristics  of  model  | 

Mine-target  sweep 
interaction,  MFI  Sonar, 
single  asset  against 
single  mine  type 

Detailed  physics 
(magnetic  acoustic 
sweep,  sonar  hunt) 
using  MC  simulation 

Large,  detailed  taking  weeks  to 
provide  results  as  cross  channel 
profile  MoP’s 

Single  Asset,  Single 
Pass,  multiple  mine 
type,  sweep  or  hunt 

Calculation  of  single 
pass  for  a  single 
asset  against 
multiple  mine 
threats  MoP 

Equation  combining  single  pass 
cross  channel  MoP  to  multiple 
mine  clearance  cross  channel  MoP 

Single  asset,  multiple 
pass,  sweep  or  hunt 

Calculation  of 
multiple  pass,  single 
asset  against 
multiple  threats  MoP 

Complex  equation  transforming 
single  pass  MoP  to  a  single  asset, 
multiple  pass  MoP  (Clearance 
plot) 

Multiple  Asset, 
multiple  pass 
combined  hunting  and 
sweeping 

Calculation  of 
combined  clearance 
for  hunting  and 
sweeping  assets 

Complex  equation  working  from 
a  plot  combining  achieved 
Clearance  from  single  assets  MoP 
to  multiple  assets  Clearance  Level 
(Combined  Clearance  MoP) 

Correlate  mines 
removed  plot  with 
Clearance  plot  to 
provide  MoE  for  threat 
to  transitor 

—  _  .  -  „  - ^  1 

Calculation  of  mines 
remaining  and 
threat  to  transitor 

Simple  (but  very  clever) 
calculation  of  MoE 

Calculation 
Done  within 
12  hour 
Tasking 
Cycle 


Submarine  Warfare  Modelling  -  Levels  of  Abstraction 


Level 


Basic  Functions:  sensors, 
weapons,  information 
management,  platform 

models _ 

CS  functions:  Detection, 
TMA,  Targeting,  SINS  POE, 


Target  engagement 


Multiple  Target 
engagement 


Task  (Described  in  DoDAF 
CORE.  Application  should 
model  defined  scenarios 
with  sufficient  accuracy) 


■I 

I 


Type  of  Model 


Detailed  physics  models, 
Integrated  Platform  System 
Model 

Single  sub-function 
performance  model  in  CORE 
including  effects  such  as 
probability  distributions  and 
computing  resources 

CS  model  execution  (prior 
example  ANZAC  Extant) 


CS  model  of  multiple  Target 
engagement  (prior  example 
I  ANZAC  ASMD) 


Defined  scenario  DoDAF 
CORE  Operational  model 


Characteristics  of  Model 


Large,  detailed,  major  effort 
to  maintain,  slow  to  generate 
results  (months)  possibly  as 
probability  or  sub-function 
EFFBD  execution  with 
internal  calculation.  A 
probability  distribution  or  a 
sub-function  model  could  be 
used  in  the  next  level. 

EFFBD  execution  with 
internal  calculation.  Output: 
probability  distribution  or  a 
sub-function  model  could  be 
used  in  the  next  level. 

EFFBD  execution  with 
internal  calculation.  Output: 
probability  distribution  or  a 
sub-function  model  could  be 
used  in  the  next  level. 

EFFBD  execution  with 
internal  calculation:  Output 
result  in  format  suitable  for 
Decision  Makers  i.e. 
Probability /Traffic  Light 
colour 


Calculation 
Done  within 
One  week 


70 


UNCLASSIFIED 


UNCLASSIFIED 


DSTO-GD-0734 


Layout  of  the  Modelling  Layers  Contained  within 
‘Clarity’  Tool  &  How  it  Can  Support  Timely  Decisions 

1 .  Detailed  models  from: 

The  Integrated  Platform  System  Model  (MPD,  DSTO)for  whole  of 
submarine  margins 

Physics  and  engineering  for  sensors  and  weapons 
With  Prior  modelling  calculation  time  -  weeks 

2.  Executable  models  in  MBSE  (CORE). 

Use  ‘distilled’  information  from  above  within  MBSE  Functions 
Submarine  functional  chain  execution  in  scenarios  &  vignettes 
Informed  by  Operations  Research 

Parametric  analysis  (minimal)  -  changes  in  few  low  level  functions 
Computation  time  -  days 

3.  Final  layout  of  results  in  formats  for  decision  makers 

May  require  information  display  tools  outside  MBSE  (CORE) 


Challenges  for  Tool  Development 

1.  Inputting  the  FSM  Project  into  MBSE 

1.1  Helpful: 

Capability  Development  using  WSAF  (MBSE  CORE) 

Should  have  two  -  five  years 

1.2  Difficult: 

Low  level  changes  to  functions  need  detailed  implementation 
-  may  be  difficult  within  Project  response  times 

2.  Moving  between  operations  and  engineering  understanding  of 

parameter  values  during  Project? 
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Engineering  vs  Operations  Understanding  of 
Parameter  Values 

1. Operational  performance  measured  from  operational/exercise 
analysis  vs. 

2. Engineering  Performance  calculated  from  physics  and 
engineering  signal  processing 


Probability 
of  Detection 


Range  from  Sensor 


Is  it  worth  doing? 

How  else  might  it  be  done? 
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